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Executive Summary 


The following study of charter schools in Missouri was conducted pursuant to 
RSMo. Section 160.410.4. The two primary components of the study are the 
evaluation of student achievement gains and a review of administrative and 
instructional best practices. 


A comparative analysis of student achievement gains from 2006 to 2009 
showed variation in performance between students in Kansas City Missouri 
School District (KCMSD) and St. Louis Public Schools (SLPS) and students in 
the charter schools in each of those cities. School effects were evaluated using 
a value-added model. In both Kansas City and St. Louis there were charter 
schools with achievement gains greater than, equal to, or less than their 
respective district averages as well as the state average. 


In addition, seven of eight secondary charter schools reported graduation rates 
higher than the graduation rates of the traditional public school district in their 
city. 


In the next section of the study, administrator and teacher interviews supplied 
the data which addressed administrative and instructional best practices. 
Across all of the interviews, autonomy, finances, and school culture were 
pervasive themes. 


Administrators and teachers spoke favorably of their ability for autonomous 
decision making and the freedom to make decisions independent from review at 
a district level. While there is no district-level accountability for charter 
schools, some administrators in charter schools managed by Education 
Management Organizations (EMOs) have less autonomy than other charter 
school leaders. Teachers also said they felt they had more autonomy primarily 
in terms of flexibility in their teaching. Throughout all the interviews, 
administrators and teachers spoke of the culture within their schools and most 
noted it as one of the strengths of their school. 


Administrators indicated that their greatest challenge and what they most wish 
they could change is funding. They mentioned a number of specific concerns 
from offering competitive teacher salaries to purchasing or expanding facilities. 


The diversity among charter schools and traditional public schools does not 


provide evidence to allow a confident or accurate assertion that one type of 
school consistently outperforms the other. 
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Section 1 — Academic Achievement 


Charter schools have been in operation in Missouri since the 1999-2000 
academic year. Currently, in Kansas City 20 charters operate on 27 campuses, 
and in St. Louis 13 charters operate on 22 campuses!. The analysis of 
academic achievement gains and reporting of graduation rates that follows 
does not include the six charter schools that opened in 2009-2010. 


Part A — Academic Achievement Growth 


Analysis was conducted by the Missouri P20 Education Policy Research 
Center?. 


This section provides an analysis of achievement growth of charter school 
students relative to achievement growth of students in traditional public 
schools in the districts where charter schools are located as well as students 
across all Missouri public schools. 


Value-added Estimates of Charter and Non-charter Public Schools in St. 
Louis and Kansas City 


Jenny Kim, Mark Ehlert, Cory Koedel, and Michael Podgursky 
Department of Economics 
University of Missouri-Columbia 


Introduction 


During the 2005-06 school year, DESE implemented a new system for 
assigning unique identifiers to all public school students (MOSIS) and began 
requiring schools to administer the MAP mathematics and communication arts 
exams to all students in grades three through eight. These changes make it 
possible to measure a student’s achievement growth over time. With the 
administration of the spring 2009 MAP exams, Missouri now has linkable test 
scores for four consecutive years. 


This study uses longitudinal MAP student achievement data from spring 2006 
to spring 2009 to compute value-added estimates of charter versus non-charter 
schools in St. Louis and Kansas City. This study is limited to schools with 
students tested in grades 3-8. 


" Source: Missouri Department of Elementary and Secondary Education. 
* University of Missouri, Truman School of Public Affairs, http://truman.missouri.edu/P20/. 
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e In Kansas City charter schools on average had larger learning gains in 
both math and communication arts than did the non-charter 
(traditional) schools in the district 

e In St. Louis charter schools had math achievement gains that were not 
significantly different from non-charter schools in the district. In 
communication arts, charter school gains were significantly less than 
non-charter schools. 

e These overall averages mask variation in the charter school sector. In 
both cities there are some charter schools that perform significantly 
above non-charters in the district. There are also some charters that 
performance significantly below. 


These are initial estimates based on the new longitudinal student data system 
in Missouri (MOSIS). Data on student achievement growth, even when 
aggregated to the school level, have considerable measurement error, and may 
exhibit some instability from year to year. As this data system matures, and 
more years of data can be averaged, the reliability of estimates of value-added 
for any school will improve. 


Data 


We retrieved MAP score records for the 2006, 2007, 2008, and 2009 testing 
cycles from the Office of Social and Economic Data Analysis (OSEDA) and 
removed records assigned to “non-regular” school districts. We conducted 
several data checking processes to identify and remove test records with 
duplicate and/or invalid student identifiers before linking records and 
calculating gain scores. We also examined the grade levels on pairs of matched 
test records to identify instances where students took the same grade level 
exam in consecutive years. Those records are excluded from descriptive 
statistics and analysis. Appendix C contains a detailed description of the 
specific data preparation steps we completed with corresponding numbers of 
MAP test records removed for various reasons. 


Analytical Framework 
Value-added models 


Our goal was to compare the performance of charter schools to traditional 
public schools in St. Louis and Kansas City. We do this by estimating a value- 
added model (VAM) for all of the schools in the state and then examining the 
estimated school effects for charter and non-charter schools in the two 
districts. 
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An important feature of VAM models is their focus on student achievement 
growth. School effectiveness is measured by the gains in student achievement 
they produce for a typical student. 


The models we estimate are presented in simplified form below: 
Gain Score = X*B1 + (school effects) + e (1) 


Two versions of the model are estimated — one for math and one for 
communication arts. The dependent variable is a gain score for an individual 
student, which is simply the spring-to-spring change in a student’s 
communication arts or math MAP scores. X is a set of control variables 
(discussed in more detail below), and B1 is a vector of coefficients that 
indicates how each control is associated with test score gains. School effects 
are a set of binary indicator variables, where the indicator for the school in 
which the student was MAP tested in the given year and grade is set to one, 
and all other school indicators are set to zero. In our model there are roughly 
1700 of these school variables. Finally, “e” is a residual that captures 
measurement error and other omitted factors that affect a student’s gain score 
that are not included in the model. 


The model is fitted to the data using Ordinary Least Squares (OLS) methods. 
The standard errors are computed in a way that takes account of clustering of 
student and school observations (i.e., there are multiple observations of the 
same student, and students within schools). Because we estimate the school 
effects from a model that predicts individual student scores, each school-effect 
estimate can be interpreted as capturing the average deviation from predicted 
student performance at that school. For example, a large positive number 
suggests that a school’s students, on average, are doing better than would 
otherwise be predicted based on prior performance and the other controls in 
our model (see below). 


The school effects that we report are adjusted so that the average school effect 
is zero statewide. Thus if a school has a significant positive value, it means that 
it performs better than the average school in the state, and vice versa for a 
negative value. 


The X*B1 term can be thought of as a filter or an adjustment factor. The idea 
is to standardize or “control for” various student factors when comparing 
schools. There are three sets of variables included in X. 

e Grade and year indicators 

e Prior year student test scores 


e Student characteristics 
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The grade and year indicators simply allow for the fact that the state-wide 
average gain scores of students are not the same from one grade to the next, 
and that the gain scores are not entirely stable from year to year. Including 
these variables has the effect of homogenizing the composition of the student 
body by grade and year in comparing one school to another (Ehlert, et.al., 
2009). 


The prior year test scores are a crucial element in the model. Although the 
dependent variable is a gain score, in fact, the gains a student can expect to 
make are also a function of the prior level of achievement in both subjects. We 
include the math and communication arts test scores from the prior year in the 
model, as well as the square of both of these scores (to capture non-linearity of 
these effects). They are consistently significant. 


Finally, we include a set of student characteristics. These include variables 
indicating whether the student was in the school less than a full school year 
(mobile student), free and reduced price lunch eligibility status, English 
language learner status (LEP), special education status, and indicators for the 
gender and race of the student. 


The estimated coefficients (B1) are presented in Appendix D. Above we noted 
that X*B1 can be thought of as a filter or adjustment. Our primary focus is in 
estimating school effects. X*B1 can be thought of a factor that homogenizes the 
composition of the student body in comparing schools. Alternatively, X*B1 can 
be thought of as a set of variables that “forecast” a student’s gain score. If 
students in school A on average do better than what we would forecast based 
on X*B1 then school A will have a positive effect. If students in school A do 
worse than forecast by X*B1 then that school has a negative effect. 


Limitations 


These estimates, while informative, should be not be taken as a definitive 
estimate of the effect of charter schools. There are a number of reasons for 
caution. The first concerns what researchers call “selection bias.” While we 
have controlled for prior test scores in both communication arts and math, and 
for a variety of characteristics of students, it may be that there are other 
factors about students that systematically differ between those who attend a 
charter school and those who attend traditional public schools. For example, 
some charter schools are explicitly targeted to “at risk” students. Presumably, 
these are students with risk factors that may not be fully captured in lagged 
test scores and other student covariates. These may be students with 
exceptionally difficult home situations (e.g., homelessness, abuse). The 
estimates we present may not fully adjust for these risk factors.* 


3 . . 73 . 0 7 
This has led some researchers to use data that approximates “natural experiments” for charter school studies. 
Hoxby and Muraka (2009) and Abdulkadiroglu (2009) in studies of charter schools in New York City and Boston use 
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A second limitation concerns duration of exposure. These estimates of school 
effects are based on one year gain-scores for students. If the effects of charter 
schools are small but accumulate over time, or require some period of time to 
begin to take effect, these one-year effects may underestimate the true effect of 
charter schools. In spite of the fact that the MOSIS system has been in place 
for several years, it is not at present possible to provide charter school 
estimates that take account of the length of exposure of the student to the 
charter school. This will be possible in future studies, however. 


Finally, we are only able to examine charter schools enrolling students in 
grades 3-8 (the MAP tested years). We could not examine the effect of charter 
schools in earlier grades, since there is no state-wide assessment given in those 
grades. However, the new End of Course (EoC) assessments were introduced in 
Spring 2009. In future years we will be able to undertake value-added 
estimates in high schools. 


Estimates 


Figures 1-3* report charter school effects in math in St. Louis. Note that these 
charter and non-charter school effects are nested in a larger model of student 
gains for all Missouri schools. In Figures 1 and 2 we report school effects for all 
Missouri public schools with at least 20 MAP scores over the period 2006- 
2009. Each dot represents an estimated school effect; 1762 school effects are 
presented. ° 


We have transformed the effects estimated in the model into standard 
deviations and normalized them so that a school that produces the average 
gain has a school effect of zero. A school with a positive effect produced student 
achievement gains that were larger than expected given the background of the 
students and prior achievement scores, and vice-versa for a negative score. 
Along the horizontal axis we present the level of student poverty or the percent 
minority enrollment in the school. 


In Figures 1 and 2 we have highlighted all of the St. Louis schools. The St. 
Louis non-charter schools are shown in green. Dark green dots are significantly 


data on queues of students applying for charter schools. These schools are oversubscribed and by law students are 
admitted based on lotteries, thus creating the potential for a randomized study. They compare achievement gains for 
students lotteried in with those who are not admitted. Both studies find large positive effects of charter schools on 
student achievement. Unfortunately, we do not have data on waiting lists for St. Louis and Kansas City. A research 
institute at Stanford University (CREDO) produced aggregated charter school estimates for 17 states using data from 
2006-2008. We have not attempted to replicate their methodology or estimates, but their findings are broadly 
consistent with this ours. For a survey of research on charter schools see Jansen and Gronberg (2009). 


* All figures referenced in this section can be found on pages 8-20. 
° Each school with 21 or more MAP scores is treated as a separate school. Some charter schools operate multiple 
campuses. If they meet the former criteria they are treated as separate schools in this analysis. 
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above or below the state average. In the case of light green dots, the school 
effect is not significantly different from the state average. The same convention 
holds for charter schools (red). Dark red dots denote schools significantly above 
or below the state average. 


In math, of the 13 charter schools for which we had more than 20 MAP gain 
scores, in 8 of thirteen cases the charters scored significantly below the state 
average. 


Figures 1 and 2 compare St. Louis charter (and non-charter) schools to the 
state average. In Figure 3 we test an alternative hypothesis by comparing St. 
Louis charter schools to the average non-charter public school. The average 
school effect in the St. Louis non-charter schools was -.13 standard deviations. 
Using this as a comparison benchmark, five of the 13 charter schools scored 
above the district average and one was insignificantly different. 


Figures 4-6 provide similar estimates for Communications Arts. Of the 13 
charter schools, one is significantly above the state average, and three are not 
significantly different. If we restrict the comparison just to St. Louis schools, 
two charters are above the St. Louis average, five insignificantly different, and 
six are significantly below. 


Figures 7-9 provide Math MAP estimates for Kansas City. In Kansas City there 
are 19 charter schools with grade 3-8 MAP scores. Five of these are 
significantly above the state average, five are statistically indistinguishable 
from the state average, and nine are significantly below the state average 
(Figures 7 and 8). If we use the Kansas City schools as the comparison group, 
six charters are significantly above the Kansas City average, five are at the 
average, and eight are significantly below the average (Figure 9). 


Figures 10-12 present Communications Arts estimates for Kansas City. Of the 
19 charter schools, four are significantly above, six are significantly below, and 
nine are not significantly different from the state average. When we switch to 
Kansas City non-charters as our benchmark, seven charters are above the 
Kansas City average, nine are not significantly different, and three are below 
the non-charter average. 


Finally, Figure 13 provides a comparison of charters as a group with non- 
charters as a group in both cities. In both cases, the comparison is with non- 
charters in the same city. In Kansas City, we find charter schools 
outperforming non-charters on both math and communication arts. The 
charter-non-charter difference is .05 and .04 standard deviations, respectively, 
in math and communication arts. In St. Louis, there is no significant difference 
between charters and non-charters in math. In communication arts, the 
average gain is .03 higher in the non-charter schools. 
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Figure 2. Standardized School Effects on MAP Math Performance 


vs. Percent Minorities 


(St. Louis) 
Dp ee re ae ere ee ae = =e 
® Other Schools 
® STL Public: Significant STL Public: Not Significant 
® STL Charters: Significant STL Charter: Not Significant 


Significant: Significantly differentfrom the statewide average of school effects. 


Standardized School Effect* 


-1.0 
0 20 40 60 80 


Percent Minorities in School 


*Standardized School Effect; = (School Effect; - Average of School Effects) /StDev of Scale Scores 
(StDev of Scale Scores = Weighted Average of by-grade Stdev of Scale Scores, for grades 3-8) 
Note: Number of schools = 1,762 (with more than 20 valid test scoresin both subjects) 





100 


Ol 


Figure 3. Standardized School Effects on MAP Math Performance 
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0 10 20 30 40 50 60 70 80 90 


Percentile Rank 


*Standardized School Effect; = (School Effect; - Average of School Effects) /StDev of Level Scores 
(StDev of Level Scores = Weighted Average of by-grade Stdev of Level Scores, for grades 3-8) 


100 


Il 


Standardized School Effect* 


Figure 4. Standardized School Effects on MAP Com Arts Performance 
vs. Percent Eligible for Free/Reduced Priced Lunch 
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Significant: Significantly differentfrom the statewide average of school effects. 
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Figure 5. Standardized School Effects on MAP Com Arts Performance 
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Figure 6. Standardized School Effects on MAP Com Arts Performance 
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Figure 7. Standardized School Effects on MAP Math Performance 
Percent Eligible for Free-Reduced Priced Lunch 
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Figure 8. Standardized School Effects on MAP Math Performance 
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Figure 9. Standardized School Effects on MAP Math Performance 
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Figure 10. Standardized School Effects on MAP Com Arts Performance 
Percent Eligible for Free-Reduced Priced Lunch 
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Significant: Significantly differentfrom the statewide average of school effects. 
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Figure 11. Standardized School Effects on MAP Com Arts Performance 
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Figure 12. Standardized School Effects on MAP Com Arts Performance 
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Figure 13. pifferencein Standardized School Effects 
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(StDev of Level Scores = Weighted Average of by-grade Stdev of Level Scores, for grades 3-8) 
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Part B -— Graduation Rates 


Eight secondary charter schools reported graduation rates for the 2008- 
2009 academic year®. Four of five Kansas City charter schools reported 
graduation rates higher than the district graduation rate for Kansas City 33 
School District, and three Kansas City charter schools have graduation rates 
higher than the state average (shown in blue in Figure 14). Of the three 
secondary charter schools in St. Louis, all three reported graduation rates 
higher than both the state average and the average for St. Louis Public Schools 


(shown in red in Figure 14). 


Figure 14. Charter School Graduation Rates 2008-2009 














Graduation Rate 








Hogan University Construction Imagine Imagine Ethel Alta Vista Don Bosco 
Preparatory Academy Careers Academy of Academy Hedgeman Charter Education 
Academy Center Careers ES&M(KC) Lyle School Center 

School 


Reference lines: St. Louis Public Schools graduation rate - 45.9%; Kansas City Public 
Schools graduation rate - 64.2%: All Missouri Schools qraduation rate - 85% 


° Two secondary schools do not have graduation rates included. Lee A. Tolbert Academy recently expanded to serve 
secondary students and currently serves through grade 11. Hope Academy in Kansas City opened in 2009-2010. 
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Section 2 — Administrative and Instructional 
Best Practices 


Administrators from 15 schools and teacher focus groups from 12 schools’ 
participated in interviews with the Executive Director of the Joint Committee 
on Education respond to questions on the administrative and instructional 
best practices at their school. The same set of questions was asked of all 
participants, and the responses are summarized below. 


Charter school administrators and teachers shared a wide range of information 
about their schools. While much of what they discussed is not uncommon or 
could be replicated in a traditional public school, other ideas reflect a unique 
environment within the charter school. Responses represent the participants’ 
perspective on the practices and culture of their school. 


Part A —- Administrator Interviews 
Why did you choose to become the principal/ director at this school? 


A few of the administrators said they accepted their current position because 
they supported the concept of charter schools. Specific comments noted an 
appreciation for the diversity of staff and students, an awareness of the 
school’s reputation, and an overall regard for the school’s culture. Nearly half 
of the administrators had been promoted from assistant principal or teacher, 
and two were former administrators in traditional public schools who came to 
charter schools after retirement. 


If you have previously been an administrator in another type of school (e.g., 
traditional public school, parochial), what ts the most significant difference 
between that work environment and your current work environment? Do you 
have opportunities or flexibility as a principal/ director in a charter school that 
were not available to you as a principal/ director in a traditional public school? If 
so, please describe. 


Several administrators had previous administrative experience outside of 
charter schools. When asked about the differences between their former work 
environment and their current work environment, the most common response 
was that they believed the charter school offered a more positive environment 
for kids than what they had experienced in traditional public schools. The next 
most common response of administrators was their ability for autonomous 
decision making in charter schools. However, this was not equal across charter 
schools as a few administrators whose schools were managed by EMOs noted 


’ At three of the schools there were not teachers available to participate in a focus group. 
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that functions of staff hiring, food services contracts, and bus contracts were 
handled by the EMO. 


Other issues administrators cited as distinct from their traditional public 
school experience were a noticeable lack of the presence of teacher 
organizations; a closer relationship with students, parents, and the 
community; and that charter schools can operate with a specific focus (e.g., 
language immersion, dropout recovery). The one negative aspect noted by 
administrators was lesser financial resources in the charter school. 


What ts the building philosophy on professional development? 


Administrators identified several characteristics of the school’s professional 
development practices although no one practice dominated the various 
responses. Some of the specific practices mentioned included individual 
teacher development plans, differentiated professional development based on 
level of experience, and professional development assistance from the 
sponsoring university. Other administrators noted support for teachers which 
included such things as university tuition reimbursements, financial support 
for National Board Certification, and mentoring for new teachers. In many 
situations, teachers had an active role in the professional development through 
leadership teams and peer teaching. Finally, a few administrators commented 
on the professional development of their school being driven by student 
achievement data and student needs and/or connected with the school’s goals. 


If there is one practice in your school that could be replicated and used to 
improve any school, what would that beP 


Of the practices administrators identified in their school as most likely to be 
able to improve any school, the most common responses centered around 
school culture, namely a strong commitment to the school’s mission by all 
staff. Most other responses were related to what the administrators noted 
regarding the school’s professional development (e.g., instructional coaching, 
mentoring). 


What ts the biggest challenge your school faces? 


Administrators noted a range of challenges faced by their school. The most 
common responses dealt with finances and facilities. A couple of the 
administrators mentioned the difficulty in offering salaries competitive with the 
local traditional school district. Aside from financial challenges, administrators 
discussed some of the challenges faced by their students: poverty, limited 
English proficiency, and poor prior academic preparation. A few of the 
administrators noted tensions with the local traditional public school district 
as a challenge. One administrator felt that he was challenged by what he saw 
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as the public perception of charter schools as a drain on traditional public 
school resources. 


If you could change one thing about your school, what would it be?P 


Increased resources was the most common response from administrators 
regarding what they would change about their school. Other changes they 
would make which are not necessarily unique to charter schools included more 
parental involvement, higher attendance, higher academic outcomes, and 
higher teacher retention. 


Part B - Teacher Focus Groups® 
Why did you choose to teach at this school? 


Teachers offered a variety of reasons for why they chose to teach at their 
school. Some mentioned simply wanted to teach in an urban setting while 
others specifically mentioned their support of the concept of charter schools. 
Teachers identified characteristics of their schools which drew them to teach 
there such as the positive culture, autonomy in teaching, the school’s mission, 
or the school’s reputation. A few teachers also identified practical reasons for 
teaching at their particular school: placement by Teach for America, a hiring 
freeze in the teacher’s home state, or being recruited to teach at the school. 


If you have previously taught in another type of school (e.g., traditional public 
school, parochial), what ts the most significant difference between that work 
environment and your current work environment? 


For teachers who had previously taught in traditional public school, they 
frequently noted their ability to make decisions about their teaching as the 
most common difference between teaching in a charter school and teaching in 
a traditional public school. Other frequent comments referenced the difference 
in culture between the teachers’ experiences in traditional public schools and 
charter schools. Teachers also noted the student focus, collegiality among staff, 
greater efficiency for meeting student needs, and the accessibility of 
administrators among the experience with charter schools. A few teachers 
commented on their perception that teachers were more motivated in charter 
schools and attributed that to a variety of reasons including generally younger 
staff in charter schools and that teachers must earn their positions without the 
security of a union. 


Aggregate data on the teachers who participated can be found in Appendix C. 
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Do you have opportunities or flexibility as a teacher in a charter school that 
would not be available to you as a teacher in a traditional public school? If so, 
please describe. 


Like the response from administrators, teachers’ favorable impression of having 
greater voice in decision making in charter schools was a common response 
when asked about opportunities afforded to them in charter schools. With this, 
most teachers were speaking about their teaching or curriculum decisions. 
Other comments reflected the same things teachers spoke of as differentiating 
their previous experience in traditional public schools from charter schools 
such as collaboration among staff, better access to administrators, and the 
opportunity to work more closely with students. 


What ts the building philosophy on professional development? What ts the 
teachers’ role in professional development? 


All of the focus groups indicated that professional development was a priority 
in their school. Most indicated that regular time was set aside for professional 
development. They echoed many of the things raised by administrators such as 
a balance between external experts or offsite training and internal peer 
coaching or individual development. A few of the teachers mentioned that 
teachers in their school had a voice in deciding the professional development 
through such things as teacher surveys or serving on professional development 
committees. In addition, several teachers mentioned that their school’s 
professional development was determined by student needs as shown through 
student achievement data. A couple of teachers noted receiving professional 
development support through their sponsoring institution or EMO. 


If there is one practice in your school that could be replicated and used to 
improve any school, what would that beP 


Of the practices in their school that teachers felt could improve any school, two 
ideas were noted most frequently: teacher empowerment in decision making 
and establishing a culture of trust in the school. When talking about teacher 
empowerment, one teacher group commented on the importance of quality 
teachers as one teacher stated, “You don’t want to empower weak teachers.” As 
for the culture of trust, teachers gave examples that affected all stakeholders in 
the school from ease of communication with administrators to positive peer 
pressure among the students to developing strong teacher-student 
relationships. Teachers also identified academic practices at their school which 
they felt would benefit any school including regular formative assessments, 
grouping students by ability/development rather than age, and the use of data 
to guide instruction. 


25 


Joint Committee on Education — Charter Schools - 2010 


What ts the biggest challenge your school faces? 


Teachers cited a variety of challenges their schools face. The most common 
challenge noted was parental involvement. Of the five teacher groups who 
mentioned this, four of them also felt the problem was exacerbated by the fact 
that their schools were not neighborhood schools. The next most common 
responses were that funding and facilities issues were a major challenge. Other 
challenges teachers mentioned were related to the students (some of which 
were unique to particular schools). Teachers identified attendance, behavior, 
academic achievement, language barriers with parents, and involuntary 
student mobility as challenges faced by their school. 


If you could change one thing about your school, what would it be?P 


Many of the teachers cited the challenges they had previously mentioned as the 
one thing they would change about their school. In addition to those issues, 
teachers said they would like to see their school have higher teacher retention. 
They also offered ideas such as keeping the school open at night for community 
functions to help the school become an anchor for the community. 
Interestingly, some teachers wished to see their schools expand while others 
wanted to see an enrollment cap or even a reduction in enrollment. 


Part C — Additional Charter School Features 


During the interviews or in information provided by the school, several unique 
features of particular schools were noted. Those include 


On-site childcare for children of teachers and staff. 

Contracts and benefits (e.g., paid vacation) for full-time substitutes. 

Full-time translators on staff to assist students and parents.? 

Extended school day and/or extended school year. 

Before-school programs beginning at 6:30 a.m. and after-school 

programs until 5:30. 

e Career-focused secondary school in close partnership with local trade 
organization. 

e French language immersion school! 


” The English language learner population at the school is 62%. 
'° The school referenced is in Kansas City. French and Spanish language immersion schools also opened in 2009- 
2010 in St. Louis. 
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Appendix A 
Missouri Charter Schools 
Enrollment 
School Opened 2009-2010 Sponsor Grades 

St. Louis 
City Garden Montessori 2008-2009 80 SLU K-4 
Confluence Academies 2003-2004 3,011 MUS&T K-10 
Construction Careers Center 2001-2002 378 SLPS 9-12 
Ethel Hedgeman Lyle 2000-2001 80/7 MBU K-12 
Imagine Academy of Academic Success 2007-2008 478 MBU K-8 
Imagine Academy of Careers 2007-2008 1,351 MBU K-12 
Imagine Academy of ES&M (StL) 2007-2008 1,339 MBU K-8 
KIPP Inspire Academy 2009-2010 83 Wash U 5 
Lift for Life Academy 2000-2001 434 SEMO 6-10 
Northside Community School 2009-2010 54 UMSL K-5 
Paideia Academy 2002-2003 505 MUS&T K-8 
St. Louis Charter School 2000-2001 913 UMSL K-8 
St. Louis Language Immersion 2009-2010 170 UMSL K-1 
Kansas City 
Academie Lafayette 1999-2000 563 UCM K-8 
Allen Village Charter 1999-2000 406 UMKC K-8 
Alta Vista Charter School 1999-2000 193 UCM 9-12 
Benjamin Banneker Charter Academy 1999-2000 344 UCM K-8 
Brookside Charter 2002-2003 490 UMKC K-11 
Della Lamb Elementary 1999-2000 453 UCM K-8 
Derrick Thomas Academy 2002-2003 901 UMKC K-8 
Don Bosco Education Center 1999-2000 250 UCM 9-12 
Frontier School of Innovation 2009-2010 303 UMKC K-5 
Genesis School Inc. 1999-2000 154 UMKC 6-9 
Gordon Parks Elementary 1999-2000 217 UCM K-5 
Hogan Preparatory Academy 1999-2000 348 UCM 9-12 
Hope Academy 2009-2010 215 UMKC 9-12 
Imagine Academy ES&M (KC) 2007-2008 1,255 MU K-12 
KIPP Endeavor Academy 2007-2008 170 MCC-PV 5-6 
Lee A. Tolbert Academy 1999-2000 624 UMKC K-12 
Pathway Academy 2009-2010 436 UMKC K-8 
scoula Vita Nuova 1999-2000 224 UCM K-8 
University Academy 2000-2001 1,085 UMKC K-12 
Urban Community Leadership Academy 1999-2000 201 UCM 5-9 

Total 2009-2010 enrollment in St. Louis charter schools 9,603 

Total 2009-2010 enrollment in Kansas City charter schools 8,832 
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Appendix B 


Attendance Rates and Reenrollment Rates 


Reenrollment Rate 


Attendance Rate 2008-2009 
School 2008-2009 (%) to 2009-2010 (%) 

St. Louis 

City Garden Montessori n/a n/a 
Confluence Academies 91.1 83.9 
Construction Careers Center 63.3 63.3 
Ethel Hedgeman Lyle 94.6 n/a"! 
Imagine Academy of Academic Success 87.8 67.0 
Imagine Academy of Careers 88.9 68.4 
Imagine Academy of ES&M (StL) 93.2 71.7 
KIPP Inspire Academy n/a n/a 
Lift for Life Academy 94.6 89.8 
Northside Community School n/a n/a 
Paideia Academy 93.1 71.7 
St. Louis Charter School 95.3 93.1 
St. Louis Language Immersion n/a n/a 
Kansas City 

Academie Lafayette 95.2 96.1 
Allen Village Charter 97.6 14.8 
Alta Vista Charter School 92.8 84.6 
Benjamin Banneker Charter Academy 89.6 59.0 
Brookside Charter 94.5 73.5 
Della Lamb Elementary 89.1 81.6 
Derrick Thomas Academy 90.1 62.9 
Don Bosco Education Center 66.0 63.7 
Frontier School of Innovation n/a n/a 
Genesis School Inc. 88.5 23.3 
Gordon Parks Elementary 94.0 65.3 
Hogan Preparatory Academy 93.8 75.1 
Hope Academy n/a n/a 
Imagine Academy ES&M (KC) 90.0 67.7 
KIPP Endeavor Academy 96.6 69.5 
Lee A. Tolbert Academy 92.9 78.3 
Pathway Academy n/a n/a 
scoula Vita Nuova 95.1 87.2 
University Academy 94.2 88.5 
Urban Community Leadership Academy 88.1 44.0 


" Ethel Hedgeman Lyle became an independent school in 2009-2010 after previously being part of another school 
managed by an EMO. 
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Charter School Attendance Rates 2008-2009 
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Reference lines: Kansas City Public Schools - 90.5% attendance rate; St. Louis Public Schools - 90.3% attendance rate; 
All Missouri schools - 95.1% attendance rate 
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Appendix C 
Data Preparation Procedures 


We retrieved individual MAP records from the data set maintained on OSEDA, 
and processed the data in four steps. Data preparation procedures were 
designed to support our ultimate goal of performing analytical examination of 
test scores and drawing inferences. We completed four data management and 
cleaning steps for the mathematics and communication arts exam scores, as 
follows: 


Step 1) Delete the records assigned to identified “special” education 
districts. 

Step 2) Identify and delete records with invalid and/or duplicate MOSIS IDs. 
Step 3) Identify and count matching records for all possible combination of 
years. 

Step 4) Identify and count matching records with proper grade increments. 


We conducted the data cleaning and preparation on all test records with a valid 
test score. 


Table 1 reports record counts for both Math and Communication Arts in each 
year, along with reductions due to data cleaning and preparation. We note that 
the number of records removed due to bad or duplicate MOSIS IDs drastically 
diminishes between 2006 and 2007, and continues to decline in 2008. (Note 
that a substantial drop in MAP counts occurs in 2009 because the 10t grade 
MAP math and the 11t® grade MAP Communication Arts exams were no longer 
mandatory.) 


Table C-1. MAP Record Counts after Data Preparation Steps 
2006 2007 2008 2009 


474,094 469,624 467,183 | 399,718 
Excluding Special Districts 472,791 468,415 466,077 399,108 
Excluding Bad Records 466,989 467,474 465,722 398,831 


a 463,866 461,311 460,466 399.413 
een Excluding Special Districts | 462,678 460,231 459,380 398.803 
Excluding Bad Records 457,105 459,340 458,962 398,524 





After removing all records with invalid and duplicate MOSIS IDs, we ended up 
with 1,799,016 test score records for 790,604 students with a unique MOSIS 
State ID and at least one Math record. For Communication Arts, we were left 
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with 1,773,931 test score records for 828,308 students with a unique MOSIS 
State ID and at least one Communication Arts test record. 


We then counted the matching records with proper grade increment. Table A-2 
presents for each subject and grade the number of matched records and the 
corresponding match rate. Since a non-missing terminal-year record (i.e., the 
second year in the paired scores) is required for analysis, we calculated the 
match rate to be the number of matched records as a percentage of the number 
of total valid records in the terminal year. As shown in the table, match rates 
are relatively constant across grade levels within any reported combination. 


Table C-2. Number of Records for the 2-Year Matched, Non-Retained in Grade 


pC NumberofMatches | Match Rate 
| | Grade | 2006-2007 | 2007-2008 | 2008-2009 | 2006-2007 | 2007-2008 | 2008-2009 | 
Math 
| 6 | 61,295 | 61,678 | 61,994 | 93.9% | 94.5% | 94.6% | 
| 8 | 64658 | 63,007 | 62,743 | 93.0% | 93.7% | 94.4% | 
Com Arts 
| 6 | 61,087 | 61,655 | 61,938 | 93.2% | 94.6% | 94.7% | 
| 8 | 64140 | 62,684 | 62,775 | 93.1% | 93.8% | 945% __| 
* Grade is terminal-year grade of each match. 
* Denominator of match rate is the number of valid records in terminal-year of each match. 
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Appendix D 


Regression Coefficients and Related Statistical Estimates 


Dependent Variable 


Math Communication Arts 


Coefficient T-Statistic Coefficient T-Statistic 

Past Scores: 
First-Year Scale Score in - 0.34194** - 30.81 0.19145** 18.91 
Math 
Squared First-Year Scale - 0.00003** - 3.37 0.00001 0.98 
Score in Math 
First-Year Scale Score in - 0.48245** - 38.19 - 0.54505** - 47.30 
Com Arts 
Squared First-Year Scale 0.00054** 95.42 0.00007** 7.84 
Score in Com Arts 

Indicators for Student 

Characteristics: 
American-Indian - 0.92858** - 2./2 0.08011 0.26 
Asian/Pacific Islander 4.64526** 26.53 1.49822** 9.38 
Black - 4.08531** - 47.43 - 1.64919** - 20.99 
Hispanic - 0.89970** - 6.46 - 0.09035 - 0.71 
Female - 2.16402** - 48.82 3.60541 ** 89.18 
Special Education - 6.16846** - 86.27 - 8.07978** -123.90 
Limited English Proficiency 0.24188 1.38 - 2.12545" - 13.32 
Free/Reduced Price Lunch - 2.37397" - 46.78 - 2.3882/** - 51.61 
Eligibility 
In the School Less Than a - 3.33210** - 29.97 - 2.63134** - 25.95 
Full School Year 

Indicators for Grade and 

Year: 
Terminal-year Grade 4 -19.46713"" -157.16 - 8.14487" - 72.10 
Terminal-year Grade 5 -18.22465** -161.81 - 6.14417** - 59.81 
Terminal-year Grade 6 -13.95812** -170.22 -16.47484** -220.29 
Terminal-year Grade 7 -17.73426** -254.94 -12.02678** -189.57 
Dummy for 2006-2007 - 0.45133** - 8.40 - 1.45330** - 29.67 
Dummy for 2007-2008 - 0.60767** - 11.47 - 0.24555** - 5.08 

R*° = 0.251 R* = 0.308 


Number of School Effects = 1,773 (except the reference school) 
Sample Size(Number of Gainscores) = 926,358 
** denotes that coefficient is significant at 1% level. 
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Appendix E 


Teacher Focus Group Characteristics 


N = 61 teachers in 12 schools 


Teaching Experience Mean Minimum Maximum 
Years teaching in current charter school 3.8 0 10 
Years of previous teaching in another setting 5.0 0 21 
Teacher Preparation n % 
Alternative or accelerated program at a college or university 10 16.4% 
Alternative certification program (e.g., ABCTE) 2 3.3% 
Alternative preparation program (e.g., Teach for America) 2 3.3% 
Content area degree, no certification 3 4.9% 
Teacher preparation program at a college or university/degree in education 42 68.9% 
Other’ 2 3.3% 
Teacher Certification n “%o 

Regular certification 48 18.7% 

Provisional certification 5 8.2% 

Temporary certification 2 3.3% 

No certification 1 1.6% 

Other’ 4 6.6% 

Level Taught n % 

Elementary 28 45.9% 

Middle 14 23.0% 

Secondary 19 31.1% 

Content Area n % 

Art 2 3.3% 

Elementary, classroom teacher 20 32.8% 

Foreign Language 4 6.6% 

Language Arts 8 13.1% 

Math 5 8.2% 

PE; 2 3.3% 

Science 4 6.6% 

Social Studies 4 6.6% 

Special Education 4 6.6% 

Specialist (e.g., library/media specialist, counselor 5 8.2% 

Previous Teaching Experience n % 

Traditional public school 25 41.0% 

Private, parochial 1 1.6% 

Private, non-parochial 1 1.6% 

Another charter school 5 8.2% 

Multiple settings 13 21.3% 

No previous teaching experience 14 23.0% 


'? One certified in occupational therapy; one with certification from an international institution. 
'> One occupational therapy license; three working toward certification requirements. 
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Appendix F 


Other Reports or Studies Including Missouri Charter Schools 


“Public School in St. Louis: Place, Performance, and Promise.” Report 
prepared by IFF and funded by the National Association of Charter 
School Authorizers. July 2009. 

http: //www.iff.org/images/IFF PDFs/STL-Report.pdf 


The report ts an analysis of the supply and demand of public school 
options by geographical areas within the city of St. Louts. 

Note: This report was funded by the National Association of Charter School 
Authorizers. 


“Multiple Choice: Charter School Performance in 16 States.” Center for 
Research on Educational Outcomes, Stanford University. June 2009. 


Full report: 
http: //credo.stanford.edu/reports/MULTIPLE_ CHOICE CREDO.pdf. 


Subsection on Missouri charter schools: 
http: //credo.stanford.edu/reports/MO CHARTER%20SCHOOL%20REP 


ORT_CREDO_2009.pdf. 
This study is a multistate analysis of student achievement gains of charter 


school students compared with their public school peers using longitudinal 
data. 


34 


Joint Committee on Education — Charter Schools - 2010 


Appendix G 


Survey Data Provided by Charter Schools 


The following schools submitted parent, student, and/or teacher survey 
they collected. 


Allen Village 

Brookside Charter School 

Derrick Thomas Academy 

Ethel Hedgeman Lyle Academy (Elementary) 

Genesis School 

Imagine Renaissance Academy of Environmental Math and Science 
(Kansas City) 

Imagine Academy of Environmental Science and Math (St. Louis) 
Imagine Academy of Careers 

Imagine Academy of Academic Success 

St. Louis Charter School 

University Academy 


Paper copies of these surveys are available for review in the office of the 
Joint Committee on Education, Rm. 502, State Capitol. 
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